Skip to content

fix: persist lifecycle backtests for drift checks - #90

Merged
Pigbibi merged 1 commit into
mainfrom
codex/quant-p0-20260710
Jul 10, 2026
Merged

fix: persist lifecycle backtests for drift checks#90
Pigbibi merged 1 commit into
mainfrom
codex/quant-p0-20260710

Conversation

@Pigbibi

@Pigbibi Pigbibi commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • switch drift-check to QuantPlatformKit reusable drift workflow
  • require lifecycle snapshot + backtest preflight before drift
  • persist walk-forward baseline backtests into lifecycle store

Validation

  • PYTHONPATH=/Users/lisiyi/Projects/_worktrees/quant_p0/QuantPlatformKit/src python3 -m pytest -q tests/test_drift_workflow_config.py tests/test_run_walk_forward_backtest.py

Dependency

  • merge after QuantStrategyLab/QuantPlatformKit reusable drift workflow lands on main

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown

🤖 Codex PR Review

Merge allowed: repeated primary findings were cleared by independent Codex arbitration

⚖️ Codex Review Arbitration

clear: BacktestOrchestrator.run() in QuantPlatformKit does not add any special persistence-only shape beyond calling persist_result(...) after an optional start_date/end_date backfill. In the current PR, the stored baseline is written via orchestrator.persist_result(baseline_raw, strategy_profile=..., domain='hk_equity', params=..., param_set_id=...), so it still receives the canonical lifecycle fields (strategy_profile, domain, param_set_id, params, param_version, run_id, source_script, computed_at) before saving. The HK runner already populates start_date and end_date itself, so the extra normalization path in run() is not materially needed here. The new test contract also checks that persisted records in the real store include _baseline_ param-set IDs and exclude _wf records, which matches PerformanceStore.load_latest_backtest() selecting baseline candidates by _baseline marker. The original concern that the persisted baseline would have a non-lifecycle shape no longer holds on this diff.

🚫 Blocking Issues

These primary findings were cleared by independent Codex arbitration:

1. 🟠 [HIGH] Logic in scripts/run_walk_forward_backtest.py

The code computes a full-window lifecycle result through scratch_orchestrator.run(...) (via_orch) and then discards it, persisting baseline_raw from a direct runner.run(...) call instead. If BacktestOrchestrator.run performs any normalization or attaches canonical lifecycle metadata before persistence, the stored baseline will not match the record shape that the reusable drift workflow later expects, leading to false drift deltas or failed baseline lookups. (line 145)

Suggestion: Persist the orchestrator-produced full-window result instead of the raw runner output, or run the full-window baseline directly against the real PerformanceStore so the exact lifecycle-compatible record is what gets stored.


Review by Codex PR Review bot • PR

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f4b226dd67

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/drift-check.yml Outdated
Comment thread scripts/run_walk_forward_backtest.py Outdated
Comment thread .github/workflows/drift-check.yml Outdated
Comment thread .github/workflows/drift-check.yml
@Pigbibi
Pigbibi enabled auto-merge July 10, 2026 17:41
@Pigbibi
Pigbibi force-pushed the codex/quant-p0-20260710 branch from 2ffd31b to e024234 Compare July 10, 2026 17:43

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2ffd31bb2f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/drift-check.yml
Comment thread pyproject.toml Outdated
@Pigbibi
Pigbibi force-pushed the codex/quant-p0-20260710 branch from 189e3f8 to 4ebc4fd Compare July 10, 2026 17:54

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4ebc4fd3b5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/run_walk_forward_backtest.py Outdated
@Pigbibi
Pigbibi force-pushed the codex/quant-p0-20260710 branch from 4ebc4fd to c777396 Compare July 10, 2026 18:02

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 38e13e39af

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/run_walk_forward_backtest.py Outdated
Comment thread scripts/run_walk_forward_backtest.py Outdated
Co-Authored-By: Codex <noreply@openai.com>
@Pigbibi
Pigbibi force-pushed the codex/quant-p0-20260710 branch from 949094e to dd5eaec Compare July 10, 2026 19:00
@Pigbibi
Pigbibi merged commit 4d1d1a7 into main Jul 10, 2026
3 checks passed
@Pigbibi
Pigbibi deleted the codex/quant-p0-20260710 branch July 10, 2026 19:05

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dd5eaecfe5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +77 to +79
if int(synthetic_days) < min_history_days:
raise ValueError(f"synthetic_days must be >= {min_history_days} for profile={params!r}")
return _runner_synthetic_market_history(days=int(synthetic_days))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject under-warmed synthetic histories

When callers set --synthetic-days to a value that is at least min_history_days but below the runner's warm-up floor, this helper now materializes exactly that short synthetic DataFrame and passes it as market_history, so HkEtfRotationBacktestRunner.run() no longer expands it with max(synthetic_days, min_history_days + 400). For example, --synthetic-days 260 passes this check and can persist an all-cash or severely under-warmed baseline; use the new MIN_SYNTHETIC_DAYS floor or preserve the runner's expansion before saving lifecycle backtests.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant